System and method for determining veracity of patient diagnoses within one or more electronic health records

ABSTRACT

A system and method, using one or more dimensions, that assesses the veracity of a diagnosis presently in a health care record is disclosed. The system assesses the veracity of a diagnosis based on a set of dimension values and a set of weightings. The system also provides a user interface to health care providers/caregivers to review the assessment made by the system and may include a feedback mechanism so that the system may adjust the weighting of the dimensions and improve the assessment of the diagnosis made by the system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to provisional application 61/789,834,filed on Mar. 15, 2013 and is herein incorporated by reference in itsentirety.

FIELD

The disclosure relates generally to electronic health records and inparticular to a system and method for verifying the accuracy of one ormore diagnoses in an electronic health record.

BACKGROUND AND SUMMARY OF THE INVENTION

Electronic health records typically contain codified diagnoses(frequently using standard coding schemes such as ICD9 or SnoMed) suchrecords may also comprise patient clinical problem lists that arerequired for billing purposes, as well as for maintaining a patientclinical problem list, for subsequent data mining, and for clinicaldecision support. In a patient data sharing healthcare ecosystem,however, there is co-mingling of diagnoses from multiple providers (whomay not know that they may all be treating the same patient), anddiagnoses from insurers, which typically include diagnoses amalgamatedfrom suppliers, laboratories, hospitals, nursing homes, imagingfacilities, pharmacies, and other entities whose products and servicesrequire a diagnosis for payment.

There is therefore a difficulty in determining in a single patient whichdiagnoses are real and/or currently active when data is shared. Theseinclude diagnoses: (1) that are presumptive and created forreimbursement of legitimate services (presumptive diagnosis), (2)diagnoses that are created for services not actually rendered (fraud),(3) diagnoses that do not represent the most severe manifestation of adisease (known as “undercoding”) because of lack of caregiver time toencode the most specific condition (e.g., plain “diabetes” vs.“Uncontrolled Type II diabetes with renal manifestations”), (4)diagnoses captured in health records by inadequately trained personnelacross the entire spectrum of caregivers (from a generalist physician,to a specialist, to a nurse, to a pharmacist, to a technical assistingwith documentation, to self-reported diagnoses by the patient), andfinally (5) diagnoses that were once true (e.g., knee sprain, influenza,or routine urinary tract infection from 1 year ago) but would beexpected to have time-expired due to the natural history of the illness,and no longer present in a patient's active diagnoses today.

In the case of presumptive diagnosis, by way of example, an imagingfacility or laboratory typically records what is known as a‘presumptive’ or ‘rule-out’ diagnosis when it submits a bill forpayment. For example, a chest x-ray is commonly associated with adiagnosis of ‘pneumonia’ when in fact that it is a presumptive diagnosisto include with the image. In many, if not most, instances when a chestx-ray is performed, pneumonia is not found, yet the presumptivediagnosis persists in the patient's medical history.

In electronic health record (EHR) sharing environments, all of thesediagnoses can become humanly impossible to sift through in the fewminutes a doctor has to treat a new patient, for example.

Therefore, it is desirable to have a system and method in which theveracity of the diagnoses (stored in a patient medical record) can bedetermined across one or more amalgamated sources of diagnosis data (anexample of an amalgamated source of diagnosis date may be amulti-provider health care record or health care records from multiplehealth care providers) at the current point of care and it is to thisend that the disclosure is directed.

Various care data concerning a patient is weighted and processed toprovide a likelihood indicator of whether the diagnosis is accurate. Theinvention can also be used to provide alternative diagnosis for a careprovider to consider, ongoing feedback may be used to enhance thereliability of the invention output.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary embodiment of the invention;

FIG. 2 illustrates a flow chart of an exemplary embodiment of theinventor; and

FIG. 3 illustrates an example of a user interface that may be displayedto a health care provider to review the veracity assessment of anembodiment of the invention.

DETAILED DESCRIPTION OF ONE OR MORE EMBODIMENTS

The system and method may be particularly applicable to a web-basedsystem that delivers diagnosis veracity assessments to one or morehealth care provider systems over the Internet and it is in this contextthat the system and method will be described. One skilled in the artwill understand that: 1) the system and method may be implemented invarious manners that are within the scope of the invention andtherefore, that the system and method are not limited to the exampleweb-based system described below; and 2) the system and method may usedifferent dimensions, different coefficients, different formulas, etcthan those described below and as such, the system and method are notlimited to the examples provided below.

In general, embodiments of the system and method, using one or moredimensions, assesses the veracity of each diagnosis found in a healthcare record when such a health care record may be an electronic medicalrecord) based on a set of dimension values and a set of weightings, andprovides a user interface to health care providers/caregivers to reviewand verify the assessment made by the system. Embodiments of the systemmay have a feedback mechanism so that the system may adjust theweighting of the dimensions to improve the assessment of the diagnosismade by the system. The system and method may have a set of dimensions,a weighting of those dimensions, and a veracity determining formulastored/available for each diagnosis (for example, diabetes, cancer,etc.). Since each diagnosis may have unique dimensions/dimension values,unique weighting and/or a unique veracity determining function that maybe fine tuned to perform the best possible assessment of the veracitydetermining formula, weighting for any particular diagnoses may beadjusted over time to fine tune the assessment. In one embodiment, thesystem and method also may infer a diagnosis for a patient based on thedimensions, the weighting of the dimensions and the function of aparticular diagnosis of a particular patient. For example, although aparticular patient has not been diagnosed with diabetes, certain tests,lab results, etc. may allow the system to infer that the patient hasdiabetes. Now, various examples of the embodiments of the system aredescribed in more detail.

FIG. 1 illustrates an example of a web-based implementation of a system109 and method for diagnosis verification of one or more diagnosis inone or more health care records of a patient that includes one or morehealth care provider units 102, such as health care provider units 102a, 102 b, . . . , 102 n, that are capable of establishing a session andcommunicating with a diagnosis veracity assessment system unit 104 overa link 110. Each health care provider unit 102 may store or have accessto a health record/electronic health record that can be communicated tothe system 109 so that the system can analyze multiple health records.The link 110 may be a wired or wireless link, such as the Internet orWorld Wide Web, cellular network, digital data network, etc., whereinthe health care provider unit(s) and the diagnosis veracity assessmentunit 104 establish a session and communicate with each other using acommunication protocol, such as HTTP or HTTPS. One skilled in the artwill realize that other protocols may be used without departing from thespirit of the disclosed invention. Embodiments of the invention are notlimited to any particular link or protocol as embodiments may usecommunications links such as landlines or cellular links, or othernetwork links, such as a local area network, wide area network, etc.

Each health care provider unit 102 may be a computerized devicecomprised of a computer processor that has sufficient processing power,memory, display capabilities and wireless/wired connectivity circuitryto interact with the diagnosis veracity assessment unit 104. Forexample, each health care provider unit 102 may be a personal computer,a terminal, a computer server of a health care provider, a multipleprovider health care record storage system, a laptop computer, or anynumber of mobile devices, a pocket PC device, a smartphone (RIMBlackberry, Apple iPhone, etc.), tablet computer, a mobile phone, amobile email device, etc. Each health care provider unit 102 may alsoinclude a typical browser software application 111, such as units 111 a,111 b, . . . , 111 n, that may be, in an exemplary web-basedclient/server implementation, a web browser (or a plurality of lines ofcomputer code stored in the health care provider unit and executed bythe processing unit of the health care provider unit) that interactswith the diagnosis veracity assessment unit 104 and generates displayimages with information from the diagnosis veracity assessment unit 104.

The diagnosis veracity assessment unit 104, in one implementation may beimplemented as one or more computer servers that may execute one or moresoftware programs. In the web-based example illustrated in FIG. 1, thediagnosis veracity assessment unit 104 may include a software-based webserver 112, such as an Apache web server (Apache Software Foundation,www.apache.org) executed by the processing unit(s) of the one or moreserver computers that establish a communications session with eachhealth care provider unit, generate the web-pages downloaded to eachhealth care provider unit 102, and receives the data/information fromeach health care provider unit. In embodiments of the invention, the webserver 112 may establish multiple simultaneous communication sessionswith a plurality of health care provider units 102. Furthermore, healthcare provider data concerning a patient diagnosis may be obtained fromhealth insurance carrier servers/databases which receive such data forreimbursement to care providers. The diagnosis veracity assessment unit104 may also be comprised of a diagnosis assessment unit 113,implemented as software instructions executed by the processing unit(s)of the one or more computer servers. The diagnosis veracity assessmentunits may generate a veracity assessment for each diagnosis in at leastone health care record for a patient. In some embodiments of theinvention, the veracity assessment units may infer new diagnosis (asdescribed herein) and communicate that assessment to one or more healthcare provider units.

The system for diagnosis verification 109 may further comprise a datastore 114, implemented as one or more databases hosted on one or moredatabase servers in the illustrated implementation. In embodiments ofthe invention, such a data store may be part of the diagnosis veracityassessment unit 104 or remotely located from the diagnosis veracityassessment unit 104. Such a data store may or may not be owned orcontrolled by the owner of unit 104. Such a data store 114 may include aplurality of health records 106 for a plurality of patients. Such healthrecords may also be stored in an electronic medical record (EMR) systemthat is remote from the diagnosis verification system 109. Such a datastore may also compare a dimensions, weighting and rule store 108 thatstores the dimensions, weighting of those dimensions, and a function todetermine the veracity for a particular diagnosis for a plurality ofdiagnosis and the assessments for each patient, for each diagnosis (andany diagnosis that may be inferred for a patient).

The plurality of health records 106 may also be one or more healthrecords from one or more health care providers (that may be stored indifferent locations) for a particular patient so that the system canretrieve all of the health care records for a patient. Access to such aplurality of records may enable the system to analyze a greater range ofdata when determining the veracity of each diagnosis in the health carerecords for a patient. In addition, access to records from each careprovider to a patient may be provided so that a current health careprovider can rapidly determine which diagnosis are correct and which arenot accurate. In addition, using the feedback (described later herein),the dimensions, weighting, and functions for any particular diagnosismay be updated and then stored in the data store 108.

The system 109 may also include a user data store 116 that may be usedto store various pieces of information about the users of the system.For example, the user data store may have a record associated with eachhealth care provider and its associated users of the system. Such arecord may comprise, for example, the preferences for each health careprovider and its associated users. Each of the units/portions of thesystem 109 may be implemented in software, hardware, or a combination ofsoftware and hardware, and embodiments of the system should not beconsidered to be limited to any particular implementation of the systemand its units.

Although a typical client/server architecture using web pages isillustrated in FIG. 1, the system can also be implemented as a hostedsystem, software as a service (SaaS) model, as a health care providerinternal system, as a stand-alone system, and other architectures as thesystem is not necessarily limited to any particular architecture orphysical implementation.

The diagnosis assessment unit 113 may comprise a veracity score unit 120that, using a predetermined set of dimensions, weighting and functions,determine a diagnosis veracity value for the diagnosis of the particularpatient. The diagnosis assessment unit may be configured to performdiagnosis veracity value calculations for a plurality of differentdiagnosis for a plurality of different patients. The diagnosisassessment unit 113 may also have a visualization dashboard generator122 that generate a diagnosis dashboard for a particular diagnosis thatmay be returned to the caregiver/health care provider who is treating apatient so that the caregiver/health care provider can view theassessment and confirm the diagnosis assessment of the system. Thesystem may receive such a user confirmation and then “learn” and improveits assessment based on such caregiver/health care provider feedback. Toaccomplish this, the diagnosis assessment unit 113 may also have afeedback processing system 124 that is closed loop and thus enablemachine learning by modifying automatically, or by user or expert input,the dimensions, weighting of the dimensions and/or function used toassess each diagnosis. Such automatic modification may also be appliedto the dimensions, weighting of the dimensions and function used toinfer a diagnosis by the system. Such feedback enables the systemassessments and inferences to be improved in part based on the caregiveror health care provider confirmation of the diagnosis assessmentsprovided by the system over time.

An example of the dimensions, weightings and functions for a particulardiagnosis whose veracity can be assessed by the system will now bedescribed. The flow chart of FIG. 2 illustrates an embodiment of amethod 140 for determining the veracity of one or more diagnoses for apatient. The following method described is to assess a particulardiagnosis for a particular patient at a caregiver site, but the sameprocess steps may be used to assess multiple diagnoses for a patient aswell as being used to assess the diagnoses for a plurality of patients.When the method is started (executed by the computerized device of thediagnosis veracity assessment unit 104 in an embodiment), a health carerecord of a patient (142) that may be in the store of the system or maybe located at a remote location/source, is retrieved. In someembodiments, multiple health care records for the patient may beretrieved from one or more health care providers. Once the health carerecord is received, the diagnosis veracity assessment unit 104 mayretrieve a first diagnosis to assess, and may also retrieve the valuesof the information in the health record (144). The system may thendetermine the diagnosis and retrieve dimensions, weighting values andfunction for each diagnosis from the store 108 (146).

Each diagnosis that is assessed by the system may be evaluated using aplurality of dimensions. In one implementation, seven dimensions may beused. The dimensions may be: (1) the type of entity which recorded thediagnosis (e.g., institution, patient, actual caregiver, and if acaregiver the level of training or specialization of the caregiver), (2)the physical location of the diagnosis (e.g., a nursing home, hospital,laboratory, imaging facility), (3) the date of the diagnosis, (4)related similar diagnoses from different entities, (5) medications takento date that suggest a particular diagnosis, (6) procedures performed todate that suggest a particular diagnosis, and (7) lab values known inthe patient's medical history that suggest a particular diagnosis.

The type of entity and physical location dimensions are hierarchicaldimensions because there is a hierarchy for the values associate withthat dimension. For example, the type of entity dimension may havevalues that are higher for a physician specialist—Endocrinologist (for adiabetes diagnosis) who is given a higher value (72, for example) than afamily practitioner (30 for example), who has a higher value than anurse (20, for example) who is given a higher value than a patient (10,for example), etc., so that the type of entity value is hierarchical andreflective of the decreasing likelihood (in this example) that each typeof entity will make the proper diagnosis. Similarly, the physicallocation dimension may have values such that a doctor's office is givena higher value (90, for example) than a nursing home (80, for example)which has a higher value than an ambulance (50, for example), which hasa higher value than a laboratory (40, for example) that has a highervalue than a medical equipment supplier (20, for example), etc., so thatthe location of diagnosis value is also hierarchical and reflective ofthe likelihood that each location will have a proper diagnosis made bythe person at such a location.

The date of the diagnosis dimension is time value with more recentdiagnosis being given a higher value to indicate that a more recentdiagnosis is more likely to be accurate. Related similar diagnoses fromdifferent entities, medications taken to date that suggest a particulardiagnosis, procedures performed to date that suggest a particulardiagnosis, and lab values known in the patient's medical history thatsuggest a particular diagnosis are given numerical values with thenumber assigned to each type of information being used to assess theveracity of a diagnosis.

Once the dimensions for a diagnosis are determined, the system retrievesthe weighting for each dimension for the particular diagnosis. Theweighing of each dimension for the particular diagnosis may have adefault value, but as was previously described, such default values maybe adjusted by the system due to closed loop feedback. The weighting ofeach dimension may be accomplished by a coefficient. In an exemplaryembodiment, the function for a particular diagnosis may have the formof:

Veracity Score for Diagnosis #1=f (a1*Dimension1*b1 Type+Count,a2*Dimension 2 b2*Type+Count, a3*Dimension3*b3 Type+Count, . . .a7*Dimension 7 b7 time)

wherein f is a function and a1, a2, a3, . . . a7 are a weightingcoefficient for each corresponding dimension and b1, b2, b3, . . . b7are each a weighting coefficient to invoke the hierarchy of theparticular dimension (if any), and “Count” represents the frequencycount of a particular variable. Thus, the final probability (or veracityscore) for a particular diagnosis may be a function of the 7 variablesand their frequency counts and the time of each event. It is understoodthat due to iterative feedback over time, the coefficients may bemodified, even empirically, such that the likelihood of the diagnosisreceiving confirmation as being correct is maximized. Embodiments of theinvention may use various different functions and should not beconsidered as limited to any particular function. For example, thesystem may use a logistic regression or other weighted, exponentialformula (asymptotic, constrains from 0 to 1 for infinite counts andvalues, etc.).

The system may determine the above described veracity scores for eachdiagnosis that is part of the health record. The system thus generates averacity score for each diagnosis (148). The visualization dashboardgenerator 122 may generate the visualization for each generateddiagnosis veracity score with the score, each dimension (and optionallya description of each dimension), and the values associated with eachdimension. An example of such a visualization dashboard is shown in FIG.3. The visualization for a particular diagnosis may be provided to ahealthcare provider/caregiver who was involved in the particulardiagnosis. The visualization dashboard may display data in a manner thatallows the healthcare provider/caregiver to view the diagnosis, thediagnosis veracity score, the dimension and the dimensions values thatresulted in the diagnosis veracity score. The visualization dashboarddisplay may also allow the caregiver/heath care provider to evaluate theveracity of the diagnosis score (true or false in the exampleillustrated in FIG. 3 at 302) which may be communicated back to thediagnosis veracity assessment unit 104 over the link. The feedbacksystem 124 receives the evaluation for each diagnosis from acaregiver/health care provider. Such feedback may then be used by thefeedback system 124 to optimize/improve the weighting coefficients forall subsequent patients who have the same diagnosis (e.g., machinelearning using multiple patient metadata) so that the diagnosis veracityassessment unit may become more accurate over time as the evaluationsare used to adjust the coefficients. For example, the feedback system124, based on the evaluation, may not modify any of the coefficientsand/or modify (increase/lower) some/all of the coefficients in order toimprove the accuracy of the veracity score for the particular diagnosis.

The visualization dashboard should provide the caregiver/health careprovider with enough information to allow such a caregiver/health careprovider to consider the data fully in order to prevent caregiver/healthcare provider bias just before casting a vote. For example, thecaregiver/health care provider may not believe that a patient hasdiabetes, however, he/she might be confronted with data (which can befrom another caregiver) that shows persistent elevated blood sugar,ophthalmic complications specific to diabetes, and/or repetitive use ofmedications that treat diabetes and resulted in a therapeutic response,such as a lowering of blood sugar values. When provided with such data,the caregiver may vote for the correctness of the diagnoses, whenwithout such information, he or she otherwise may not have done so. Byway of example, a table similar to what is shown in the visualizationdashboard illustrated in FIG. 3 may be constructed for each potentialdiagnosis.

In addition to the generating of the diagnosis veracity score asdescribed above, the system may also identify previously unidentifieddiagnosis in a health record. In particular, based on the samedimensions, weighting and formulas described above, the system can infercertain diagnoses of a patient's condition. For example, the system mayretrieve data from the patient's health care record that indicates apersistent elevated blood sugar, ophthalmic complications specific todiabetes, and/or repetitive use of medications that treat diabetes andresulted in a therapeutic response, such as a lowering of blood sugarvalues. These values of the dimensions can be used by the system toinfer that the patient has diabetes even though there is not a diabetesdiagnosis in the patient's health record. For example, if the veracityscore for a particular previously unreported diagnosis is about apredetermined threshold (such as 90%), then the diagnosis becomes aninferred diagnosis. Conversely, patient data trending in the oppositedirection would indicate that a diabetes diagnoses is incorrect. Whilethe foregoing has been with reference to a particular embodiment of theinvention, it will be appreciated by those skilled in the art thatchanges in this embodiment may be made without departing from theprinciples and spirit of the disclosure, the scope of which is definedby the appended claims.

The invention claimed is:
 1. A system for optimizing a veracity scorefor a particular diagnosis of a plurality of diagnoses associated with apatient, the system comprising: a plurality of healthcare providerunits, each of which comprise a display; a store that stores, for eachof the plurality of diagnoses associated with the patient, a set ofweighting coefficients, a set of dimensions, and a function; a computerserver comprising executable software instructions, which when executedconfigure the computer server to: retrieve a health record of a patientfrom at least one of the plurality of healthcare provider units andgenerate a diagnosis veracity score for each of the plurality ofdiagnoses contained in the health record of the patient, wherein theplurality of diagnoses comprises the particular diagnosis; for theparticular diagnosis: retrieve from the store a set of values, whereineach of the set of values is associated with one of the set ofdimensions for the particular diagnosis from the health record, whereineach of said set of values is selected to reflect the likelihood thatthe associated dimension in the set of dimensions represents a reliableindicator of the particular diagnosis, said set of dimensionscomprising: the type of entity which recorded the diagnosis, the date ofthe diagnosis, medications taken to date that suggest the particulardiagnosis, and lab values that suggest the particular diagnosis;retrieve from the store the set of weighting coefficients for theparticular diagnosis wherein each dimension has a weighting coefficientassociated with it, and wherein each of the weighting coefficients isselected to reflect the likelihood that the associated dimension of theset of dimensions is a reliable indicator of the particular diagnosisrelative to other dimensions in the set of dimensions; retrieve from thestore the function for the particular diagnosis, wherein said functioncomprises said weighting coefficients and dimensions; determineautomatically a veracity score for the particular diagnosis bymultiplying each of the retrieved weighting coefficients with theassociated retrieved dimension to arrive at a value and applying each ofthe values to the retrieved function, wherein said veracity score is anumerical percentage value; generate, for the particular diagnosis, avisualization dashboard for display on the display of at least one ofthe healthcare provider units, said visualization dashboard comprising:a table comprising: a set of dimensions for the particular diagnosis;and a set of values for the set of dimensions for the particulardiagnosis; wherein each of the set of dimensions for the particulardiagnosis are associated within the table with one of the set of values;the particular diagnosis veracity score; and a caregiver evaluation toolfor providing feedback regarding the particular diagnosis veracityscore, wherein said caregiver evaluation tool comprises a prompt forreceiving a user selection of true or false; and receive a caregiverevaluation comprising the user selection of true or false by way of thecaregiver evaluation tool and automatically optimize the set ofweighting coefficients for the particular diagnosis by modifying one ormore of the set of weighting coefficients which are subsequently storedin the store based upon the received caregiver evaluation to improve theaccuracy of the veracity score; wherein the function for the particulardiagnosis is a logistic regression, a weighted function, or anexponential function.
 2. The system of claim 1, wherein the computerserver executes software instructions to infer a new diagnosis notcontained in the health record based on the veracity score of the newdiagnosis.
 3. The system of claim 1, wherein the computer serverexecutes software instructions to determine the veracity of eachdiagnosis of the patient contained in all health records for a patient.